The use of measure functions for evaluating classifier

Evaluation of classifier performance is often based on statistical methods e.g. cross-validation tests. In these tests performance is often strongly related to or solely based on the accuracy of the classifier on a limited set of instances. The use of measure functions has been suggested as a promising approach to deal with this limitation. However, no usable implementation of a measure function has yet been presented. This article presents such an implementation and demonstrates its usage through a set of experiments. The results indicate that there are cases for which measure functions may be able to capture important aspects of the evaluated classifier that cannot be captured by crossvalidation tests. Index TermsClassifier performance, cross-validation, data mining, evaluation, machine learning